Replace deprecated compression options in cql #4689
+27
−8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: toom <thomas@strangebee.com>
To-om
force-pushed
the
cassandra5-compress-options
branch
from
3694c4a
to
4b7cc43
Compare
Signed-off-by: toom <thomas@strangebee.com>
|
ScyllaDB still uses the old options from Cassandra 2. Therefore, two different table initialization methods are needed to account for this difference. The incompatibility of compression options has been reported here: |
porunov
reviewed
Oct 11, 2024
| @@ -331,7 +331,7 @@ private static CreateTableWithOptions compressionOptions(final CreateTableWithOp | |||
| int chunkLengthInKb = configuration.get(CF_COMPRESSION_BLOCK_SIZE); | |||
|
|
|||
| return createTable.withOption("compression", | |||
| ImmutableMap.of("sstable_compression", compressionType, "chunk_length_kb", chunkLengthInKb)); | |||
| ImmutableMap.of("class", compressionType, "chunk_length_in_kb", chunkLengthInKb)); | |||
There was a problem hiding this comment.
Some of the solutions for compatibility I think:
- We can fetch Cassandra version before creating any tables (here):
select release_version from system.local. I didn't check if ScyllaDB returns different versioning then Cassandra. If it returns only numbers then it's not useful too much, but if it returns something that defines it uniquely then we can determine if we are working with Cassandra 3, Cassandra 4, Cassandra 5, or ScyllaDB. In this case, if it's Cassandra and the version is >= 5 then we use new definition. Otherwise, we use previous definition. - We can provide an optional parameter which forces usage of one or another naming regardless of what version we are working with. If so, users can specify if they want to use
sstable_compressionorclass. As such, they can work with Cassandra 5 or any other storage which has new names for options.
There was a problem hiding this comment.
The latest version of ScyllaDB (6.2) returns the 3.0.8 when requesting the release_version. Selecting the right compression options based on this information seems to be a good option.
This reverts commit 885456f. Signed-off-by: toom <thomas@strangebee.com>
Signed-off-by: toom <thomas@strangebee.com>
porunov
reviewed
Oct 27, 2024
| @@ -437,7 +437,7 @@ CQL storage backend options | |||
| | storage.cql.compaction-strategy-options | Compaction strategy options. This list is interpreted as a map. It must have an even number of elements in [key,val,key,val,...] form. | String[] | (no default value) | FIXED | | |||
| | storage.cql.compression | Whether the storage backend should use compression when storing the data | Boolean | true | FIXED | | |||
| | storage.cql.compression-block-size | The size of the compression blocks in kilobytes | Integer | 64 | FIXED | | |||
| | storage.cql.compression-type | The sstable_compression value JanusGraph uses when creating column families. This accepts any value allowed by Cassandra's sstable_compression option. Leave this unset to disable sstable_compression on JanusGraph-created CFs. | String | LZ4Compressor | MASKABLE | | |||
| | storage.cql.compression-type | The compression class value JanusGraph uses when creating column families. This accepts any value allowed by Cassandra's compression class option. Leave this unset to disable compression class on JanusGraph-created CFs. | String | LZ4Compressor | MASKABLE | | |||
There was a problem hiding this comment.
build-doc CI fails because in the last sentence here you specified compression class but in configuration option only compression is specified. You can execute mvn --quiet clean install -DskipTests=true -pl janusgraph-doc -am to automatically re-generate this file, so that compression class automatically turns into compression.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes discussion #4688
For all changes:
issuediscussion associated with this PR? Is it referenced in the commit message?issuediscussion number you are trying to resolve?master)?For code changes:
For documentation related changes: